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Summary 


Despite  shrinking  budgets,  the  U.S.  military  is  struggling  to  simulta¬ 
neously  fund  force  levels,  current  operations,  and  an  aggressive  mod¬ 
ernization  program.  Many  believe  the  military  can  fund  its 
recapitalization  program  if  cost  efficiencies  can  be  achieved  from 
within  infrastructure  budgets.  One  way  to  reduce  infrastructure  costs 
is  through  competition,  outsourcing,  and  privatization.  Whether  the 
in-house  (or  organic)  team  or  the  private  team  wins  the  contract,  the 
government  benefits  because  the  competition  lowers  costs  and 
increases  productivity. 

Outsourcing,  or  more  precisely,  the  competition  and  cost  visibility 
brought  about  by  outsourcing,  has  been  a  successful  weapon  in  the 
fight  to  reduce  costs  [1] .  Warfighters  want  to  know:  “Will  we  have  the 
support  to  complete  our  mission?”  The  current  infrastructure  gets 
the  job  done.  Will  squeezing  the  infrastructure  for  savings  put  the 
Navy  at  too  high  a  risk?  To  meet  its  mission,  the  fleet  must  have 
proper  training,  qualified  personnel,  superior  equipment,  and  other 
infrastructure  products.  These  products  need  not  be  provided  wholly 
by  government  employees. 

This  paper  examines  the  maintenance  of  the  Navy’s  TA-4Js.  The  value 
of  this  analysis  is  that  it  allows  us  to  look  at  a  long  series  of  perfor¬ 
mance  and  cost  data,  both  for  in-house  and  contractor  maintenance. 
Because  we  have  data  on  three  contactors,  we  can  also  examine  the 
effect  of  changing  contractors. 

The  Navy  has  flown  A-4  Skyhawk  light  attack  aircraft  since  the  1950s. 
For  more  than  25  years,  the  Naval  Aviation  Training  Command  has 
used  a  two-seat  version  of  the  jet,  the  TA-4J,  to  train  student  naval  avi¬ 
ators  in  the  advanced  phase  of  strike  training.  Until  the  mid  1980s, 
the  Navy  used  its  own  personnel  and  equipment  for  the  organiza- 
tional-(O-)  and  intermediate-(I-)  level  maintenance  of  the  aircraft.  At 
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that  point,  the  decision  was  made  to  open  the  maintenance  to 
private  companies. 

Lockheed  won  the  first  competition  in  1985  for  O-level  mainte¬ 
nance  with  a  bid  that  was  about  20  percent  lower  than  the  in- 
house  offer.  In  1989,  Burnside  OTT  won  the  contract  for  I-level 
maintenance.  In  1990,  Grumman  replaced  Lockheed  as  the  con¬ 
tractor  for  O-level.  Then  in  1993,  O-level  and  I-level  maintenance 
were  combined  into  one  contract  for  both  the  TAJ  and  the  T-2 
and  awarded  to  UNC.  This  fixed-price  contract  consists  of  1  base 
year  and  4  option  years.  Finally,  in  1995  Sabreliner  was  awarded 
the  contract  for  depot-  (D-)  level  maintenance  of  both  platforms. 
The  TA-4J  portion  of  this  contract  was  estimated  at  about  $11.3 
million. 

We  analyzed  a  range  of  data,  but  we  focused  on  direct  mainte¬ 
nance  man-hours  per  flight  hour  (DMMH),  full  mission  capable 
rate  (FMC),  and  mission  capable  rate  (MC).  DMMH  is  one  mea¬ 
sure  of  the  labor  input,  and  MC  and  FMC  are  both  measures  of 
performance. 

Two  themes  emerged  from  the  analysis.  First,  contractors  per¬ 
formed  better  than  the  Navy  in-house  team  in  almost  every  case. 
For  both  maintenance  man-hour  and  FMC  rates,  the  contractor 
means  were  considered  “better.”  The  means  of  the  MC  rates  for 
both  the  Navy  in-house  team  and  the  contractors  were  about 
equal.  This  seems  to  show  that  the  contractors  were  able  to 
achieve  a  similar  MC  rate  and  a  better  FMC  rate  than  those  of  the 
in-house  Navy  maintenance  team — and  do  it  in  fewer  man-hours. 
Here,  outsourcing  results  in  the  same  or  better  quality  product 
and  increased  efficiency. 

The  second  finding  shows  that  at  the  start  of  the  contract,  perfor¬ 
mance,  as  measured  by  the  maintenance  statistics,  declined.  In 
almost  all  cases,  the  contractor’s  initial  performance  was  worse 
than  that  of  the  in-house  team,  or  it  exceeded  the  in-house  Navy 
standard  only  to  deteriorate  thereafter.  On  average,  it  was  about  2 
years  before  the  contractor  began  to  improve.  This  deterioration 
of  performance  occurred  only  when  the  maintenance  was  trans¬ 
ferred  from  the  Navy  to  the  contractor.  Thereafter,  when 


contractors  changed,  quality  did  not  suffer.  This  suggests  that  the 
problem  lay  with  transitioning  from  Navy  personnel  to  civilians.  After 
that,  when  the  contract  changed  hands,  only  the  management  team 
changed.  Because  the  workers  remained  in  place,  the  benefit  of  their 
experience  was  retained,  and  the  transition  was  smooth. 

In  this  case,  competition/ outsourcing  worked.  The  contractors 
improved  the  availability  of  aircraft  to  the  training  command  while 
using  resources  more  efficiently.  This  does  not  prove  that  the  in: 
house  Navy  system  was  inefficient,  but  the  competition  from 
outsourcing  allowed  any  weaknesses  to  be  exposed  and  resolved.  The 
downside  of  this  case  is  that  the  contractors  took  a  long  time  to 
recover  from  the  declines  in  the  maintenance  rates  during  the 
transition. 
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Introduction 


During  this  period  of  tight  budgets  and  government  downsizing,  the 
Department  of  Defense  has  looked  closely  at  outsourcing  and  privati¬ 
zation  as  a  means  to  reduce  infrastructure  costs.  To  achieve  the 
desired  levels  of  ships,  aircraft,  and  troops,  and  overall  moderniza¬ 
tion,  supporting  infrastructure  must  cost  less  and  be  more  efficient. 
Outsourcing  and  privatization  are  methods  of  moving  functions  once 
done  by  the  government  to  the  private  sector,  which  can  save  money 
and  resources. 

Even  if  outsourcing  makes  more  sense  fiscally,  from  the  warfighters’ 
point  of  view,  service  quality,  not  cost,  is  paramount.  The  warfighters 
view  the  supporting  infrastructure  not  as  a  logical  place  to  save 
money,  but  as  a  resource  that  gives  commanders  the  means  to  per¬ 
form  their  missions,  and  ultimately,  to  fight  and  win  wars.  Will  reduc¬ 
ing  the  infrastructure  by  outsourcing  strengthen  or  weaken  our 
warfighting  ability?  Will  spending  less  on  infrastructure  mean  the 
troops  must  fight  with  inferior  weapons,  training,  and  equipment  or 
will  it  give  them  more  resources  to  do  their  job? 

In  this  paper,  we  analyze  a  particular  outsourcing  experience  to  deter¬ 
mine  whether  outsourcing  works  in  a  practical  sense;  that  is,  does  the 
military  really  get  an  equivalent  output?  This  is  not  a  cost  comparison; 
it  is  a  comparison  of  productivity. 

According  to  the  Defense  Science  Board  Task  Force  on  Outsourcing 
and  Privatization,  all  supporting  activities  ashore  can  be  outsourced 
[2].  The  services  have  known  for  a  long  time  that  savings  can  result 
from  competing  and  outsourcing  functions  to  the  private  sector.  In 
the  mid  1980s,  for  example,  the  Navy  turned  over  the  maintenance  of 
aircraft  in  the  training  command  to  private  contractors.  Here  we  con¬ 
centrate  on  one  specific  area  of  that  conversion.  We  attempt  to 
quantify  the  changes  and  compare  the  effectiveness  and  efficiency  of 
Navy  and  contracted  maintenance. 
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Background 

The  Navy  trains  its  aviators  in  phases  at  a  number  of  locations.  All 
future  pilots  begin  primary  flight  training  in  a  fixed-wing  aircraft,  the 
T-34C  Turbo  Mentor.  Upon  completing  this  22-week  program,  stu¬ 
dent  pilots  are  divided  into  one  of  three  pipelines  for  further  train¬ 
ing:  strike  (jets),  rotary  wing  (helicopters),  or  maritime  (propeller). 
Those  students  selected  for  strike  are  sent  to  intermediate  training  at 
either  NAS  Kingsville  in  Texas,  or  NAS  Meridian  in  Mississippi  for  a 
23-week  course  flying  the  T-2C  Buckeye.  Those  who  qualify  then  take 
advanced  strike  training,  which  is  also  given  at  Kingsville  and  Merid¬ 
ian.  On  successful  completion  of  this  25-week  advanced  course,  the 
students  earn  their  wings.  Until  recendy,  these  students  received  all 
their  advanced  training  on  the  TA-4J  Skyhawk. 

A-4  Skyhawks  are  single-seat,  single-engine  light  attack  jet  aircraft 
flown  by  the  United  States  Navy  and  Marine  Corps  and  a  number  of 
foreign  military  services.  They  first  came  into  service  in  the  1950s  as 
nuclear-weapon-capable  strike  aircraft  and  were  flown  extensively 
during  the  Vietnam  conflict  in  a  conventional  attack  role.  Highly 
regarded  for  their  exceptional  handling  qualities,  a  two-seat  version 
was  adapted  to  serve  as  the  Navy’s  advanced  jet  training  aircraft.  The 
Navy’s  Flight  Demonstration  Squadron  (the  Blue  Angels)  flew  Sky- 
hawks  for  many  years  as  well.  The  last  production  aircraft  was  com¬ 
pleted  in  February  1979  and  delivered  to  the  Marine  Corps.  The 
airframe  has  been  retired  from  its  primary  role  as  an  attack  aircraft, 
but  some  are  still  flown  in  the  fleet,  primarily  as  adversary  aircraft  at 
commands  such  as  the  Fighter  Weapons  School  (Top  Gun)  and  com¬ 
posite  squadrons,  where  they  fill  various  roles.  The  TA-4J  version  still 
serves  in  the  advanced  training  command  at  NAS  Meridian,  but  it  is 
gradually  being  replaced  by  the  T-45  Goshawk. 

Before  the  mid  1980s,  all  the  Navy’s  O-  and  I-  level  maintenance  on 
the  TA-4J  aircraft  was  done  in-house.  That  is,  Navy  enlisted  personnel 
performed  all  required  work.  Here  we  refer  to  this  as  the  “in-house” 
case. 
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As  a  result  of  an  A-76  competition,  the  Navy  outsourced  the  maintenance 
of  the  aircraft.  The  contractor’s  winning  bid  was  about  20  percent  lower 
than  the  in-house  bid,  after  taking  contract  management  and  competi¬ 
tion  costs  into  account.  The  work  was  transferred  to  the  contractor  one 
level  at  a  time,  starting  in  1985.  The  overall  list  is  stylized  in  figure  1.  The 
first  contract  was  awarded  to  Lockheed  for  O-level  support  in  1985,  and 
the  contract  switched  to  Grunman  in  1990.  Burnside  OTT  won  the  first 
contract  for  I-level  support  in  1989.  In  1993,  the  contracts  for  O-level  and 
I-level  were  combine  into  one  contract  awarded  to  UNC.  Finally,  D-level 
support  was  awarded  to  Saberliner  in  1995. 


Figure  1 .  TA-4J  contract  maintenance  award  dates 


Year 

O- Level 

1- Level 

D-Level 

1980 

ORGANIC 

1985 

1986 

1987 

1988 

1989 

1990 

1991 

1992 

1993 

1994 

1995 

Lockheed 

Burnside  OTT 

Grumman 

UNC 

Saberliner 

A  previous  CNA  study  [3]  found  that  the  material  readiness  of  surface 
ships  after  depot  maintenance  was  about  the  same  whether  the  work  was 
done  in  a  public  (Navy)  yard  or  a  private  yard.  Our  work  complements 
that  research.  Here  we  examine  O-  and  I-  level  maintenance  for  aviation. 
O-level  maintenance  can  be  completed  by  the  squadron  (or  its  contrac¬ 
tor  equivalent)  and  consists  of  such  tasks  as  troubleshooting  discrepan¬ 
cies,  basic  engine  and  transmission  repair,  and  replacing  parts.  I-level 
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maintenance,  the  next  step  up,  is  completed  by  the  Aircraft  Interme¬ 
diate  Maintenance  Department  (AIMD). 

We  chose  the  A-4  for  this  comparison  because  we  have  ample  data  for 
much  of  the  time  period  that  the  maintenance  was  done  by  Navy  per¬ 
sonnel,  and  from  the  time  contractors  took  over  maintenance  until 
the  present  day. 

We  collected  all  our  data  from  the  CNA  Aviation  Information  Digest 
(AID)  database,  which  is  a  compilation  of  the  NAMSO  4790.A7936", 
Aviation  3-M  Data  Report.  We  took  the  training  data  directly  from 
training  squadron  reports  before  maintenance  was  shifted  to  the  con¬ 
tractor  and  from  training  wing  reports  after  maintenance  was  out¬ 
sourced. 

The  3-M  data  are  maintenance  statistics  reported  monthly  by  all  avia¬ 
tion  commands  to  the  Navy  Aviation  Maintenance  Support  Office.  All 
data  are  compiled  from  Maintenance  Action  Forms  (MAFs) ,  which 
are  comprehensive  accounts  of  all  maintenance  completed  on  each 
aircraft  in  a  command.  Examples  of  the  statistics  include  percent  full 
mission  capable  (FMC),  cannibalizations  per  100  flight  hours,  and 
total  number  of  AIMD  parts  processed.  These  data  are  a  known  value 
throughout  the  fleet,  are  common  across  communities,  and  are  easily 
accessible.  To  address  quality,  we  examined  the  following  O-level  data 
for  the  period  from  April  1980  until  January  1996: 

•  FMC:  Percent  of  time  the  aircraft  is  fully  ready,  with  zero  system 
degradation.  Higher  rates  for  this  category  are  more  desirable, 
meaning  the  aircraft  are  fully  ready  a  larger  percentage  of  the 
time. 

•  Mission  capable  (MC) :  Percent  of  time  the  aircraft  can  fly  and 
complete  a  mission.  It  is  more  inclusive  than  FMC,  and,  as  with 
FMC,  higher  values  are  more  desirable. 

•  Direct  maintenance  man-hours  per  flight  hour  (DMMH):  The 
amount  of  O-level  maintenance  completed  for  every  flight 
hour  flown.  Lower  rates  can  show  greater  efficiency. 

The  expected  effect  of  competition/outsourcing  is  an  increase  in  effi¬ 
ciency  (also  termed  productivity).  This  is  either  an  increase  in  the 
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performance  available  at  any  particular  cost  or  a  decrease  in  the  cost 
of  attaining  a  specified  performance.  Figure  2  shows  this  expected 
effect. 


Figure  2.  Competition/outsourcing  increases  efficiency 


cost 


Whether  this  increased  efficiency  will  result  in  lower  costs,  higher  per¬ 
formance,  or  some  combination  of  the  two,  depends  in  part  on  how 
the  contract  is  written. 


Analysis 


The  first  data  set  we  looked  at  was  direct  maintenance  man-hours 
(DMMH)  per  flight  hour,  as  shown  in  figure  3.  We  use  the  DMMH 
rate  as  a  proxy  for  costs.  The  initial  DMMH  rates  for  the  contractor 
rose  sharply  as  the  contract  went  into  effect.  This  rise  may  indicate 
that  the  contractor  had  to  negotiate  a  "learning  curve"  or  “break-in” 
period  at  the  start  of  the  contract.  Yet  even  with  this  initial  adjust¬ 
ment,  the  change  in  DMMH  rate  seems  to  be  significant  and  an 
improvement  over  the  in-house  Navy  maintenance.  This  would  sug¬ 
gest  that  the  contractors  could  ready  the  aircraft  to  fly  equivalent 
hours  more  efficiendy  (i.e.,  with  fewer  personnel) .  The  question  then 
becomes:  If  the  contractor  used  fewer  man-hours  than  the  in-house 
maintenance  team,  was  overall  readiness  affected? 


Figure  3.  O-level  DMMH  per  flight  hour 
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One  way  to  measure  the  learning  curve  or  break-in  period  is  to  deter¬ 
mine  how  long  it  took  the  contractor  to  reach  the  mean  level  for  that 
rate.  We  will  see  that  the  mean  DMMH  rate  under  contractor  mainte¬ 
nance  was  9.60,  and  that  it  took  the  contractor  55  months  to  reach 
this  level.  Even  so,  the  contractor’s  first  efforts  were  better  than  the 
in-house  DMMH  rate.  Also,  it  took  the  contractor  only  about 
21  months  to  arrest  the  rising  trend  in  contract  DMMH  and  begin  to 
lower  it  to  more  efficient  rates. 

Although  we  based  all  our  conclusions  on  month-by-month  data,  the 
graphs  shown  here  are  smoothed  average  annual  numbers,  which 
eliminated  the  jumps  in  the  plots  one  might  expect  with  the  large 
standard  deviations  encountered. 

Turning  to  mission  capable  rates,  the  generally  positive  trend  drops 
off  with  the  start  of  contract  maintenance  (figure  4) .  The  rate  even¬ 
tually  climbs  to  a  level  similar  to  when  the  Navy  was  doing  the  main¬ 
tenance.  But  is  it  statistically  significant? 


Figure  4.  MC  rates 
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The  break-in  period  for  MC  rates  was  nearly  as  long  as  for  direct  main¬ 
tenance  man-hours.  The  contractors  took  49  months  to  reach  the 
mean  MC  rate  of  65.17  percent.  In  this  case,  the  contractor  also  has 
some  ground  to  make  up  to  reach  the  level  of  the  in-house  team’s  MC 
rate.  However,  the  contractor’s  rate  began  to  improve  after  about  29 
months. 

The  MC  rate  is  the  only  3-M  data  set  that  is  specified  in  the  actual 
maintenance  contract.  Contractor  performance  which  does  not  meet 
an  MC  rate  of  65  percent  for  the  TA-4J  for  3  consecutive  months  is 
reason  to  end  the  contract.  (The  two  other  rates  that  the  contractor 
is  obliged  to  meet — aircraft  ready  for  issue  at  55  percent  and  sortie- 
completion  rate  at  92  percent — are  not  included  in  3-M  statistics.) 

The  FMC  rate  graph  (figure  5)  shows  an  overall  improvement  both 
for  the  Navy  maintenance  and  the  contract  maintenance.  After  recov¬ 
ering  from  an  initial  downturn,  the  rate  for  contract  maintenance 
eventually  surpasses  that  of  in-house  Navy  maintenance. 


Figure  5.  FMC  rate 
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It  took  the  contractor  41  months  to  reach  a  mean  value  of  59.81  per¬ 
cent  for  FMC.  About  29  months  elapsed  before  the  contractor’s  rate 
began  to  improve. 

Table  1  shows  the  basic  information  for  the  formal  hypothesis  test 
conducted  on  the  data  collected  for  the  in-house  Navy  and  contractor 
maintenance  on  the  TA-4J  Skyhawks  of  the  training  commands. 


Table  1 .  Results  of  z-test 


Mean 
Std.  dev. 

Z  critical  (RR  bound) 

Test  statistic  (z) 
Conclusion 


DMMH 

Contract  Organic 
96  14.52 

3.71  2.37 

-1.64 
Lower  tail 
-1 1 .68 
Reject  H0 


MC 

Contract  Organic 
65.17  66.15 

8.88  6.9 

1.64 

Upper  tail 
-0.90 

Accept  H0 


FMC 

Contract  Organic 
59.81  53.47 

10.05  8.85 

1.64 

Upper  tail 
4.89 

Reject  Hc 


The  test  rejects  the  null  hypothesis  that  the  DMMH  means  are  equal 
in  favor  of  the  alternative  hypothesis  that  the  contract  DMMH  mean 
is  less  than  the  in-house  DMMH  mean. 

The  test  accepts  the  null  hypothesis  that  the  MC  means  are  equal  for 
both  the  in-house  and  contract  cases.  The  test  rejects  the  null  hypoth¬ 
esis  concerning  the  FMC  rates.  Instead,  it  accepts  the  alternative 
hypothesis  that  the  contract  FMC  mean  is  greater  than  the  in-house 
FMC  mean. 

This  suggests  that  the  contractors  had  a  better  FMC  rate  than  the 
Navy.  However,  for  MC  rates,  the  Navy  and  the  contractor  performed 
about  the  same.  The  DMMH  rate  is  interesting.  The  conclusion  is  that 
the  contractors  had  a  better  DMMH  rate  than  the  in-house  Navy 
team.  In  other  words,  the  contractors  used  fewer  maintenance  man¬ 
hours  to  complete  a  flight  hour  than  the  in-house  Navy  team  did.  This 
means  that  the  contractors  had  the  training  aircraft  ready  to  fly  at 
rates  that  equaled  or  exceeded  the  Navy’s,  and  the  contractor  did  it 
with  fewer  maintenance  man-hours. 
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Although  we  believe  that  DMMH,  FMC,  and  MC  capture  the  sub¬ 
stance  of  the  comparison,  we  analyzed  other  SM  data  categories  as 
well.  Another  data  set  that  can  be  used  to  estimate  quality  of  mainte¬ 
nance  is  the  “not  mission  capable”  (NMC)  rate.  In  one  sense,  this  data 
set  is  the  opposite  of  mission  capable.  (Either  the  aircraft  is  mission 
capable  or  it  is  not  mission  capable.)  However,  NMC  status  is  subdi¬ 
vided  by  cause:  not  mission  capable  due  to  supply  reasons  (NMCS)  or 
maintenance  reasons  (NMCM) .  An  example  of  NMCS  would  be  a 
part  is  needed  but  is  not  available,  whereas  an  aircraft  would  be 
NMCM  if  the  parts  are  available  but  the  person  with  the  correct  skills 
is  not.  To  complicate  things  further,  an  aircraft  can  accumulate 
NMCM  time  and  NMCS  time  concurrently.  Figure  6  shows  the  NMC 
data  for  the  periods  in  question. 

Figure  6.  NMCM  rate 
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Notice  that  the  contract  NMCM  rate  immediately  jumped  to  a  much 
higher  level  than  the  in-house  rate,  and  then  continued  to  climb.  This 
is  an  excellent  example  of  the  learning-curve  effect.  Figure  7  shows 
the  results  from  the  NMCS  data. 


Although  the  contract  rate  for  NMCS  again  jumps  from  the  in-house 
level,  the  contractor  makes  a  steady  and  dramatic  improvement,  even 
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Figure  7.  NMCS  rate 


Year 


when  compared  with  the  low  rates  of  the  last  years  of  in-house  main¬ 
tenance.  In  fact,  the  improvement  in  the  NMCS  rate  allows  the  con¬ 
tractor  to  match  the  overall  NMC  rate  of  the  in-house  years.  Table  2 
summaraizes  the  NMC  data. 


Table  2.  Highlighting  the  NMC  rate 


NMCM  NMCS  NMC  total 

Organic  Contract  Organic  Contract  Organic  Contract 

Mean 
Std.  dev. 

Zcritical  (RR  bound) 

Test  statistic  (z) 

Conclusion 

The  z-test  for  the  NMC  statistics  gives  some  interesting  results.  The 
test  is  a  lower  bound  case  because  the  alternative  hypothesis  states 
that  the  contract  means  are  less  than  the  in-house  means.  In  other 
words,  to  support  the  alternative  that  the  contract  means  are  “better” 
than  the  in-house  means,  we  want  the  test  statistic  to  be  sufficiently 


I 


3.76  5.66 

-1.64 
Lower  tail 
2.49 

Cannot  reject  H0 


12.26  10.87 

3  .'26  4.59 

-1.64 
Lower  tail 
-2.52 
Reject  H0 


34.57  34.83 

6.40  8.88 

-1.64 
Lower  tail 
0.24 

Accept  Hc 
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negative  to  be  below  the  critical  value,  and  therefore  reject  the  null 
hypothesis.  If  the  opposite  were  true,  and  the  in-house  means  were 
greater  than  the  contract  means,  then  the  test  statistic  could  be  posi¬ 
tive  (or  at  least,  not  sufficiently  negative)  and  support  the  null.  This 
is  true  in  the  case  of  NMCM.  The  test  statistic  is  2.49  with  the  critical 
value  at  1.64,  and  normally  we  could  accept  the  null  hypothesis  that 
the  contract  and  organic  means  are  equal.  Intuitively,  however,  it  is 
hard  to  make  that  claim.  The  contract  mean  is  ’’worse”  than  the  in- 
house  mean  and  has  a  higher  variance.  If  the  alternative  hypothesis  is 
altered  to  a  two-tailed  test  by  stating  that  the  contract  and  in-house 
means  are  not  equal  (as  opposed  to  saying  one  is  greater  or  less  than 
the  other),  the  critical  value  changes  to  ±1.96,  and  the  test  statistic 
falls  in  the  upper  portion  of  the  rejection  region.  For  this  reason,  and 
to  keep  the  test  consistent  with  the  rest  of  the  study,  the  conclusion 
for  NMCM  as  stated  in  table  2  is  that  we  cannot  reject  the  null  hypoth¬ 
esis  that  the  contract  and  the  in-house  means  are  equal. 

The  NMCS  rate  is  much  more  straightforward.  As  table  2  clearly 
shows,  the  test  statistic  falls  in  the  rejection  region,  and  we  can  reject 
the  null  hypothesis  that  the  contract  and  in-house  means  are  equal  in 
favor  of  the  alternative  that  the  contract  NMCS  mean  is  less  than  (bet¬ 
ter  than)  the  in-house  NMCS  mean. 

The  overall  result  is  that  the  supply  portion  of  the  NMC  rate  indicates 
that  the  contractor  rates  were  better,  yet  the  maintenance  portion  of 
the  NMC  rate  seems  to  show  that  in-house  personnel  did  a  better  job 
than  the  contractors.  Putting  the  two  halves  together  and  recalculat¬ 
ing  the  test  implies  that  the  two  portions  of  the  overall  rate  were  close 
enough  to  wash  out  any  advantage  one  had  over  the  other,  leaving  the 
final  conclusion  that  the  in-house  NMC  rate  equaled  that  of  the  con¬ 
tractor.  This  is  consistent  with  the  results  from  table  1.  Remember 
that  not  mission  capable  and  mission  capable  are  opposites,  so  if  we 
determined  one  data  set  to  be  equal,  we  would  expect  the  other  to  be 
equal  as  well. 

Why  the  two  components  of  NMC  differed  is  harder  to  discern.  Why 
would  the  contractor  have  a  higher  NMC  rate  due  to  maintenance 
factors?  One  possibility  is  that  the  contractor  did  not  initially  hire 
enough  workers  to  do  the  job  in  a  timely  fashion,  or  did  not  hire  work- 
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ers  who  had  the  needed  training.  The  other  possibility  is  that  the  con¬ 
tractor  did  not  start  out  with  the  right  support  equipment. 

We  could  argue  that  to  keep  mission  capable  aircraft  flying,  the  con¬ 
tractors  cannibalized  parts  from  other  NMC  aircraft.  Cannibalization 
is  the  practice  of  taking  supply  parts  off  an  aircraft  that  is  “down”  (not 
mission  capable)  and  putting  them  on  another  aircraft  that  may  need 
only  one  or  two  parts  to  keep  it  flying.  It  is  tracked  as  the  number  of 
cannibalizations  per  100  flight  hours.  A  high  cannibalization  rate  may 
result  from  inefficiencies  in  supply  or  maintenance  administration. 
Figure  8  shows  that  at  first,  the  contractor  did  little  cannibalizing. 
Eventually,  the  contractor  cannibalization  rate  climbed,  but  then  it 
leveled  out  and  soon  declined  to  a  rate  that  was  lower  than  it  had 
been  at  the  start.  The  mean  rate  for  the  contractor  was  5.26,  whereas 
the  mean  rate  for  the  in-house  team  was  7.56.  Although  the  contrac¬ 
tor’s  rate  did  climb,  it  never  approached  the  high  mark  for  cannibal¬ 
ization  under  in-house  maintenance. 


Figure  8.  Cannibalizations 
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Intermediate-level  maintenance 

Most  of  the  previous  data  has  dealt  with  O-level  maintenance.  The 
contract  for  the  TA-4J  also  covers  I-level  maintenance,  which  is  work 
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done  by  an  Aviation  Intermediate  Maintenance  Department  (AIMD) . 
Three  statistics  give  some  insight  into  this  level  of  work.  When  a 
“gripe”  (defect)  is  written  against  an  aircraft  or  component,  one  of 
three  things  happen: 

•  The  gripe  is  repaired. 

•  The  gripe  cannot  be  fixed  at  that  level  of  maintenance.  (It  is 
beyond  the  capability  of  maintenance  (BCM).) 

•  No  defect  is  found  (or  the  problem  can’t  be  attributed  to  the 
part  being  worked  on) . 

Figure  9  shows  the  percent  of  AIMD  items  processed  that  were 
repaired. 

At  first  glance,  the  contractor  and  the  in-house  rates  appear  to  be 
heading  in  opposite  directions.  We  used  a  linear  regression  to  con¬ 
duct  a  time  trend  analysis  with  inconclusive  results.  Even  when  we 
included  an  apparently  significant  break-in  period,  the  contractor 
mean  for  AIMD  parts  repaired  was  59.9  versus  a  rate  of  59.4  for  the 
in-house  team.  These  are  statistically  equal. 


Figure  9.  Percentage  of  parts  repaired  by  AIMD 
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The  second  statistic  gives  the  percentage  of  AIMD  items  processed 
that  were  declared  BCM,  that  is,  items  that  had  to  be  passed  on  for 
depot-level  repair.  Figure  10  shows  this  information.  Again,  the  con¬ 
tractor  appears  to  have  higher  rates  compared  to  in-house  mainte¬ 
nance.  With  a  mean  of  31.6,  in-house  maintenance  clearly  has  a  lower 
rate  than  the  contractor  mean  of  34.8,  and  in  this  case,  the  z-test  bears 
this  out.  For  the  scope  of  this  study,  the  BCM  rate  might  be  the  least 
meaningful.  We  assume  that  many  items  processed  as  BCM  is  a  bad 
thing,  indicating  an  inability  to  do  the  work  on  a  lower  level.  This  may 
not  be  the  situation  at  all.  By  the  time  the  contractors  took  over  the 
work,  the  aircraft  fleet  had  aged  considerably,  and  may  well  have  had 
many  defects  requiring  depot  repair.  In  the  last  years  of  in-house 
maintenance,  there  was  also  an  upturn  in  the  mean  AIMD  BCM  rate, 
which  may  support  this  idea. 

The  third  statistic  is  the  percentage  of  AIMD  items  processed  with  no 
defect.  (See  figure  11.)  Finding  that  the  part  processed  wasn’t  the 
cause  of  the  original  gripe,  or  that  the  part  processed  wasn’t  defective 
are  two  reasons  why  a  gripe  may  be  in  this  category.  Although  both 
in-house  and  contract  rates  were  low,  the  contractor  rate  showed  vast 
improvement  throughout  the  time  period.  As  the  hypothesis  test 
bears  out,  the  contract  rate  for  AIMD  no  defects  is  statistically  lower 
than  the  in-house  rate. 

These  data,  although  interesting,  do  not  make  a  simple  comparison 
for  quality  purposes.  I-level  maintenance  data  do  not  easily  relate 
directly  to  the  problem  of  keeping  aircraft  flying.  Nor  does  the  con¬ 
tract  parameter  dealing  with  I-level  relate  well  to  the  3-M  statistics  we 
are  using.  (The  contract  calls  for  a  90-percent  ready-for-issue  rate  for 
aircraft  components  inducted  for  I-level  maintenance  for  which  Cl  or 
C3  capability  exists  as  determined  by  the  Activity  Individual  Compo¬ 
nent  Repairability  List  (ICRL).)  Because  no  data  set  we  looked  at  in 
relation  to  I-level  favored  either  the  contract  or  the  in-house  style  of 
maintenance,  the  best  we  can  say  is  that  not  much  distinguishes  the 
two,  and  ultimately,  contract  I-level  maintenance  is  at  least  as  good  as 
in-house  maintenance. 


Figure  1 0.  Percentage  of  AIMD  items  processed  that  were  BCM 


Year 


Figure  1 1 .  Percentage  of  AIMD  items  processed  with  no  defect 


Year 


Break-in  period 

We  assumed  a  break-in  period,  or  learning  curve,  would  be  necessary 
when  the  contractor  first  took  over.  In  most  of  the  data  sets  we  ana¬ 
lyzed,  there  indeed  seemed  to  be  a  period  at  the  start  of  contract 
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maintenance  when  maintenance  rates  trended  down  from  levels 
achieved  under  in-house  maintenance.  For  example,  the  number  of 
contractor  maintenance  man-hours  started  low,  but  then  climbed 
rapidly  to  a  peak  before  falling  to  a  low  value  once  again.  Explana¬ 
tions  for  this  phenomenon  could  include  inexperienced  personnel, 
insufficient  staff,  incorrect  or  insufficient  support  equipment,  or  an 
increase  in  the  amount  of  work  required.  Once  they  were  past  this  ini¬ 
tial  learning  curve,  the  contractors  consistently  improved  mainte¬ 
nance  rates,  even  when  the  contract  was  recompeted  and  changed 
hands.  Anecdotal  evidence  suggests  that  this  might  have  occurred 
because  the  same  workers  remained  when  the  contractors  changed, 
but  few  government  employees  joined  the  initial  contractor.  In  this 
case,  those  government  employees  were  mostly  enlisted  Navy  person¬ 
nel  who  were  ordered  to  sea-going  billets. 

To  determine  how  the  break-in  period  affected  the  comparison 
between  in-house  and  contract  maintenance,  we  removed  those  par¬ 
ticular  months  from  the  contractor’s  dates  and  did  the  comparison 
once  more.  Table  3  shows  the  results  from  this  round  of  testing. 


Table  3.  Results  of  z-testing  with  break-in  periods  removed 

DMMH  MC  FMC 


Contract 

Organic 

Contract 

Organic 

Contract 

Organic 

Mean 

9.5 

14.5 

68.2 

66.2 

63.4 

53.5 

Std.  dev. 

3.96 

2.37 

7.57 

6.9 

8.12 

8.85 

Z  critical  (RR  bound) 

-1.64 

Lower  tail 

1.64 

Upper  tail 

1.64 

Upper  tail 

Test  statistic  (2) 

-10.63 

1.90 

7.92 

Conclusion 

Reject  H0 

Reject  Hc 

Reject  H0 

When  we  remove  the  break-in  periods,  we  reduced  the  number  of 
observations  from  115  to  94  for  the  DMMH  rate  and  to  86  for  both 
the  MC  and  the  FMC  rates.  The  in-house  observations  were  left  at  98. 
These  reductions  reflect  the  months  until  the  rates  began  to  improve. 
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As  we  would  expect,  the  contract  numbers  improve  for  all  three  rates. 
The  standard  deviations  for  the  contract  FMC  and  MC  rates  were 
reduced  as  well. 

In  the  end,  this  leads  to  no  real  change  for  the  DMMH  and  FMC  con¬ 
clusions.  The  mean  rates  for  the  contractor  are  “better”  than  the  cor¬ 
responding  mean  rates  for  the  in-house  team. 

In  the  original  analysis,  the  conclusion  was  that  mean  MC  rates  for  the 
contractor  and  the  in-house  team  were  equivalent.  Eliminating  the 
break-in  period  does  improve  the  contractor’s  mean  MC  rate  which 
rises  to  68.2  (from  65.2). 

However,  statistical  tests  do  not  support  a  conclusion  that  contractor 
rates  are  better  than  MC  rates  attained  by  the  in-house  maintenance 
team.  The  conclusion  can  be  drawn  that  the  contractor’s  mean  MC 
rate  is  at  least  as  good  at  the  in-house  team’s. 
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Productivity 

Figure  12  shows  the  ratio  between  full  mission  capable  and  direct 
maintenance  man-hour  rates.  It  shows  how  much  maintenance  was 
needed  for  each  percentage  point  of  full  mission  capable  status.  If; 
for  example,  we  can  consider  the  FMC  rate  as  an  output  and  the 
DMMH  as  an  input,  the  ratio  is  similar  to  what  economists  call  labor 
productivity. 


Figure  12.  Estimate  of  labor  productivity 


25 
20 

1  5 

FMC/DMMH 

1  o 

5 
0 

8  08  08  1  8  2  8  28  3  8  4  8  4  8  58  6  8  68  78  88  88  9  9  09  09  1  929  2  9  3  9  49  49  5 
4128  41284128412841284128  41284128 

Date 


During  the  in-house  period  of  maintenance  from  early  1980  to  mid 
1988,  the  ratio  was  fairly  steady  at  around  4  or  5  to  one,  or,  one  main¬ 
tenance  man-hour  led  to  4  to  5  percentage  points  of  FMC  status. 
When  maintenance  was  first  contracted  out,  the  ratio  increased 
briefly  to  about  10,  then  fell  to  the  in-house  level.  After  a  period  of 
“learning,”  the  ratio  started  to  rise,  eventually  reaching  about  15  by 
1995. 
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The  evidence  suggests  that  the  contractors  experienced  a  break-in 
period  at  the  start  of  the  contract.  After  about  2  years,  they  reversed 
the  downward  trend  and  began  to  improve.  Just  how  the  contractors 
managed  this  reversal  is  still  unknown,  but  possible  explanations 
include  capital  improvements  to  the  physical  plant,  increased  train¬ 
ing,  and  increased  hiring.  Again,  it  is  striking  that  no  break-in  period 
is  evident  through  the  remaining  years  of  the  contracts,  denoting  a 
smooth  turnover  between  contractors.  Probably,  this  is  because  con¬ 
tract  workers  remained  when  a  new  contractor  took  over.  In  the  end, 
the  contractor  was  getting  at  least  twice  the  FMC  rate  for  every  man¬ 
hour  of  maintenance  completed  versus  the  organic  maintenance. 
Contract  maintenance  seems  to  be  more  efficient. 

Figure  13  tells  much  the  same  story  as  figure  12,  but  in  absolute 
numbers. 


Figure  13.  FMC  versus  DMMH 
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Again  we  see  that  when  the  contractors  assumed  the  maintenance  of 
the  TA-4Js  in  the  training  command,  a  higher  FMC  rate  eventually 
emerged,  and  the  maintenance  man-hour  rate  declined.  For  now,  we 
can  only  guess  at  the  reasons.  It  may  be  that  because  the  workers  often 
remain  in  place  and  do  not  rotate  as  Navy  technicians  do,  they  gain 
more  experience  and  efficiency.  Or  it  may  be  that  the  contractors 


greatly  improved  the  capital  of  the  facilities.  Whatever  the  reason,  the 
contractor  seems  to  be  more  efficient,  and  is  putting  out  a  “more 
ready”  product  at  a  lower  man-hour  cost. 

One  obvious  oudier  in  the  contract  data  corresponds  to  September 
1990  and  is  characterized  by  a  very  high  DMMH  rate.  The  FMC  rate 
for  that  month  is  unremarkable;  however,  both  the  sortie  rate  (the 
number  of  flights)  and  the  utilization  rate  (the  number  of  flight 
hours  per  airframe)  for  the  month  are  unusually  low.  September 
marks  the  end  of  the  fiscal  year,  and  anecdotal  evidence  suggests  that 
the  training  command  may  have  run  out  of  money  to  fly  that  month. 
Yet  no  other  September  in  the  data  set  jumps  out  like  September 
1990.  (September  1990  was  also  the  second  month  of  Desert  Shield. 
But  neither  August  nor  October  1990  show  any  unusual  variations 
from  other  years.) 

Labor  productivity  versus  technological  advances 

Throughout  this  study,  we  have  contended  that  we  can  measure  qual¬ 
ity  and  cost  of  maintenance  from  the  3-M  statistics  and  that  one 
method  (contract  or  in-house)  could  be  rated  as  “better.”  We  must 
understand,  however,  what  this  classification  does  and  does  not  mean. 

Because  the  Navy  had  a  higher  DMMH  rate  than  the  contractor  does 
not  necessarily  mean  Navy  maintenance  personnel  were  inefficient. 
For  the  capital  outlay  the  Navy  put  into  the  facilities,  the  in-house 
maintenance  team  may  have  been  working  at  peak  efficiency.  With  no 
regard  for  cost,  the  Navy  may  have  been  content  to  operate  with  out¬ 
dated  equipment  and  facilities,  as  long  as  it  was  meeting  the  mini¬ 
mum  requirements. 

Private  firms,  on  the  other  hand  will  not  operate  a  plant  than  cannot 
cover  costs.  They  have  an  incentive  to  ensure  the  plant  is  running  at 
peak  efficiency,  even  if  that  means  paying  for  the  most  advanced  tech¬ 
nology.  The  goal  is  to  have  a  plant  that  provides  the  greatest  surplus 
over  operating  costs.  Loosely  defined,  this  is  “best  practice.”  Private 
firms  will  gravitate  to  a  best  practice  plant  much  faster  than  the  Navy 
could  or  would.  In  other  words,  it  is  possible  that  the  contractors 
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increased  productivity  as  a  result  of  technological  improvements  in 
the  facilities  and  not  because  in-house  maintenance  was  inefficient. 

From  an  economist’s  standpoint,  this  may  well  be  true.  The  view  of 
the  customer,  however,  might  be  much  less  specific.  Outsourcing  and 
privatization  save  money  because  they  create  competition  and  cost 
visibility.  The  result  is  that  the  contractor  has  an  incentive  to  lower 
input  costs  and  the  customer  receives  better  productivity,  either 
through  increased  output  or  decreased  costs. 


Conclusions 


When  the  transition  from  in-house  to  contractor  maintenance  began, 
most  3M  data  rates  got  worse.  The  contractors  took  almost  4  years  to 
reach  the  previous  mean  levels  and  2  years  before  they  showed  any 
appreciable  improvement.  Thus,  for  2  to  4  years,  the  training  com¬ 
mand  suffered  reduced  MC  and  FMC  rates. 

In  the  long  run,  outsourcing  the  maintenance  function  in  the  train¬ 
ing  command  has  not  hurt  the  quality  of  maintenance,  nor  has  it 
affected  readiness.  Because  of  unanswered  questions  regarding  labor 
productivity,  we  cannot  say  now  that  the  contractors  are  “better”  than 
in-house  maintenance,  but  we  can  say  that  once  the  break-in  period 
is  over,  the  level  of  quality  provided  by  the  contractors  is  at  least  equal 
to  that  of  the  previous  in-house  team. 

Finally,  the  contractors  used  far  fewer  resources,  as  measured  in  main¬ 
tenance  man-hours,  to  complete  the  job.  The  contractor  provided  an 
equivalent  flight  hour  with  a  33-percent  reduction  in  direct  mainte¬ 
nance  man-hours,  an  obvious  resource  and  cost  savings.  Some  of 
these  gains  could  have  been  used  to  increase  MC  and  FMC.  Outsourc¬ 
ing  leads  to  a  gain  in  efficiency.  How  this  gain  is  divided  between  per¬ 
formance  gains  and  cost  reductions  depends  on  how  the  contract  is 
written. 
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Appendix:  Methodology 

We  want  to  compare  the  3-M  data  for  both  in-house  and  contract 
maintenance  completed  in  the  training  command.  By  organizing  the 
data  and  making  two  simple  calculations  (mean  and  standard  devia¬ 
tion),  we  see  that  the  data  sets  appear  to  be  different.  We  want  to 
know  whether,  over  hundreds  of  observations,  the  data  sets  are  “close 
enough”  to  say  they  are  the  same,  or  if  they  are  different  enough  to 
say  one  data  set  is  better  than  the  other.  To  accomplish  this  goal,  we 
used  a  standard  z-test. 

The  objective  of  a  statistical  test  is  to  challenge  a  theorized  value  for 
a  population  parameter.  To  understand  how  it  works,  we  must  first 
define  some  terms: 

•  Alternative  hypothesis  (Ha) :  the  research  theory  we  wish  to  sup¬ 
port,  normally  by  showing  that  the  converse  to  the  alternative, 
the  null  hypothesis,  is  false. 

•  Null  hypothesis  (H0) :  a  statistical  theory  to  be  tested  and  then 
accepted  or  rejected  in  favor  of  the  alternative. 

•  Test  statistic:  the  formulation  used  to  test  the  null  hypothesis. 
This  is  what  does  the  work  of  the  test.  For  this  study,  it  will  be 
the  z-test. 

•  Rejection  region  (RR) :  specifies  the  values  for  which  the  null 
hypothesis  will  be  rejected  or  accepted. 

As  an  example  of  the  process,  consider  the  following  hypothetical  sit¬ 
uation.  A  poll  is  taken  among  a  random  sample  of  1,000  Navy  enlisted 
personnel,  surveying  their  satisfaction  with  bell-bottom  dungarees. 
Forty-seven  percent  of  those  polled  say  they  favor  wearing  bell-bot¬ 
toms.  Can  we  say  that  the  majority  of  Navy  enlisted  personnel  dislike 
bell-bottoms?  In  this  case,  taking  50  percent  as  the  point  where  we 
reach  a  majority,  we  can  say  our  null  hypothesis  is  that,  at  a  minimum, 
50  percent  of  enlisted  Navy  personnel  favor  bell-bottoms,  that  is, 


31 


H0:p>  0.5.  If  we  disagree  with  this  assumption,  our  alternative  hypoth¬ 
esis  could  be  that  the  percentage  of  enlisted  personnel  Navy-wide  who 
prefer  bell-bottoms  is  actually  less  than  50  percent,  or,  Ha:p<0.5. 
Because  the  sample  size  is  large,  we  used  the  z-test  for  the  test  statistic, 
although  no  formal  proof  is  offered  here.  We  determined  the  rejec¬ 
tion  region  by  choosing  how  much  error  we  were  willing  to  accept  in 
our  calculations.  Here  again  we  need  a  few  definitions: 

•  Type  I  error:  Rejecting  the  null  hypothesis  when  it  is  actually 
true.  The  probability  of  a  type  I  error  occurring  is  represented 
by  a. 

•  Type  II  error:  Accepting  the  null  hypothesis  when  the  alterna¬ 
tive  hypothesis  is  actually  true.  The  probability  of  this  occurring 
is  represented  by  p. 

Note  that  a  and  P  are  inversely  related;  that  is,  as  a  increases,  P 
decreases,  and  vice  versa.  The  key,  then,  to  balancing  the  proportion 
is  to  ensure  that  the  sample  size  is  sufficiendy  large  to  best  explain  the 
population  as  a  whole.  In  general  terms,  as  the  sample  size  increases, 
P  decreases  for  a  fixed  a. 

Once  the  sample  size  is  determined,  the  most  common  method  of 
determining  the  rejection  region  is  to  fix  the  probability  of  a  type  I 
error  and  look  up  the  boundaries  in  a  table.  With  the  rejection  region 
determined,  we  can  compute  the  test  statistic,  and  declare  one  of 
three  outcomes.  If  the  test  statistic  falls  within  the  rejection  region,  we 
can  say  we  reject  the  null  hypothesis  in  favor  of  the  alternative.  If  the 
test  statistic  does  not  fall  in  the  rejection  region  and  the  value  of  P  is 
sufficiendy  low,  we  can  say  we  accept  the  null  hypothesis.  However,  to 
determine  p,  we  must  state  a  specific  value  for  the  alternative,  and  this 
can  be  cumbersome.  Even  if  we  suggest  a  valid  alternative,  calculating 
the  probability  of  a  type  II  error  can  be  difficult.  Therefore,  if  the  test 
statistic  does  not  fall  within  the  rejection  region  and  we  cannot  deter¬ 
mine  the  value  of  P,  or,  if  we  can  and  the  value  is  large,  we  then  say 
that  we  fail  to  reject  the  null  hypothesis,  and  can  try  to  get  additional 
information  before  declaring  a  conclusion. 

For  the  case  of  A-4J  Skyhawk  maintenance  in  the  training  command, 
we  wish  to  test  whether  the  means  of  each  contract  maintenance  data 


set  are  equal  to  the  means  of  each  corresponding  in-house  mainte¬ 
nance  data  set.  If  both  data  sets  are  equal,  then  subtracting  one  from 
the  other  should  yield  zero.  This  will  be  the  null  hypothesis.  Alterna¬ 
tively,  we  wish  to  test  whether  the  means  of  the  contract  maintenance 
data  set  are  better  than  those  of  the  organic  data  set.  Because  we  have 
large  sample  populations,  we  will  use  the  z-test  as  the  test  statistic.  The 
rejection  region  will  be  defined  using  0.05  as  our  value  for  a,  the 
probability  of  making  a  type  I  error. 


H0:  (IV  |I0)  =0 

Ha:  (pc-  pQ)  >  0  for  MC  and  FMC,  or 
(IV  p0)  <  0  for  DMMH 

^  „  (Yc-Yo)-D0 

Test  statistic:  Z  =  _ 


Rejection  region:  z  >  za  (upper  tail  for  MC  and  FMC),  or 
z  <  za  (lower  tail  for  DMMH) 

Where 


Pc 

Po 


(Yc-Y0) 


D0 

2 


nc,n0 


the  mean  values  for  each  contract  maintenance  data  set 
the  mean  values  for  each  organic  maintenance  data  set 
the  point  estimator  of  the  difference  of  means 

the  hypothesized  difference  of  means,  set  at  0  in  this  case 
the  respective  contract  and  organic  population  variances 
and 

the  respective  contract  and  organic  sample  sizes. 


We  use  both  the  lower  tail  and  upper  tail  tests  because  the  alternative 
states  that  the  contract  data  means  are  “better”  than  the  organic 
maintenance  data  means.  For  FMC  and  MC,  this  means  that  the  con¬ 
tract  means  are  believed  to  be  greater,  so  the  upper  tail  derivative  is 
suitable.  On  the  other  hand,  a  lower  direct-maintenance-per-flight 
hour  rate  is  believed  to  be  better,  so  we  use  the  lower  tail  test. 
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